Weesp is easy to reach, and easy to leave. The last train back to Amsterdam is at 00.37. If you came from Amsterdam, you can combine a stop in Weesp with a trip to Naarden, which has much larger 17th-century fortifications. Take the train to Naarden-Bussum station 4 trains per hour, next stop, 7 minutes, from there it is 30 minutes walk to the bastions and old town centre. You do not need an extra ticket to stop off at intermediate stations. Naarden and Muiden can also be reached by bus 110 from Weesp station, every half hour.
There is one other bus line you can use: bus 122, run by BBA (http://www.bba.nl). It runs through the small villages south of Weesp, to Overvecht station in the north of Utrecht. Despite the proximity to Amsterdam, this is a less developed rural area, otherwise unserviced by public transport. Hourly, 30 or 32 minutes past the hour, until 18.30.
Note: There are also regular bus connections every half hourdirectly to several Amsterdam metro stations. I took this bus in September 2009.
